Effective Parallel Computing via a Free Stale Synchronous Parallel Strategy
نویسندگان
چکیده
منابع مشابه
More Effective Distributed ML via a Stale Synchronous Parallel Parameter Server
We propose a parameter server system for distributed ML, which follows a Stale Synchronous Parallel (SSP) model of computation that maximizes the time computational workers spend doing useful work on ML algorithms, while still providing correctness guarantees. The parameter server provides an easy-to-use shared interface for read/write access to an ML model's values (parameters and variables), ...
متن کاملProfit-effective parallel computing
R esearchers widely use speedup, efficiency, and scalability1–3 to assess parallel-computing performance. These metrics encourage researchers to use any novel technique to design or improve a parallel system, without paying enough attention to the cost increase that such a technique incurs. However, as national-defense applications are downsizing, commercial applications are the dominant users ...
متن کاملOn Convergence of Model Parallel Proximal Gradient Algorithm for Stale Synchronous Parallel System
With ever growing data volume and model size, an error-tolerant, communication efficient, yet versatile parallel algorithm has become a vital part for the success of many large-scale applications. In this work we propose msPG, an extension of the flexible proximal gradient algorithm to the model parallel and stale synchronous setting. The worker machines of msPG operate asynchronously as long a...
متن کاملScientific Computing on Bulk Synchronous Parallel Architectures
Bulk synchronous parallel architectures offer the prospect of achieving both scalable parallel performance and architecture independent parallel software. They provide a robust model on which to base the future development of general purpose parallel computing systems. In this paper we theoretically and experimentally analyse the efficiency with which a wide range of important scientific comput...
متن کاملScientiic Computing on Bulk Synchronous Parallel Architectures
Bulk synchronous parallel architectures o er the prospect of achieving both scalable parallel performance and architecture independent parallel software. They provide a robust model on which to base the future development of general purpose parallel computing systems. In this paper we theoretically and experimentally analyse the e ciency with which a wide range of important scienti c computatio...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Access
سال: 2019
ISSN: 2169-3536
DOI: 10.1109/access.2019.2936820